This invention relates to a method of, and a system for, routing messages to multiple consumers. The invention provides a system for message dispatch in a managed environment.
An asynchronous messaging system handles requests, which represent inbound work, for a server complex running service instances. Requests are received by the central processor which can examine them and dispatch them to service instances (consumers). The central processor analyses each incoming request and assigns it a classification that specifies the service level with which it should be processed. The service level has a goal, such as maximum response time, and the central processor attempts to ensure that the request is processed in accordance with the goal. The central processor also monitors the loads (for example, processor load, memory consumption and response times) on the backend servers and dispatches requests to backend servers in an attempt to ensure that no backend server is over or under burdened with work. The central processor is attempting to satisfy both these goals.
If the protocol used to convey the requests is synchronous, the implementation of the above system is relatively straightforward. When classifying a request message, the central processor can decide to which consumer (service) the message should be directed, and dispatch/invoke that consumer (service). Examples of synchronous protocols include RPC, HTTP and IIOP.
However, when applied to an asynchronous protocol, a straightforward implementation will run into problems in attempting to satisfy the dual goals of message service level and server workload profile. A straightforward implementation based on the above only has loose coupling between the classification logic and the dispatch logic. There are two known loosely coupled implementations, each of which has drawbacks. One implementation performs eager assignment of a consumer instance; the other implementation performs lazy assignment. In the eager assignment implementation, the received message is examined on arrival, the message handler decides which classification fits the message and identifies a consumer instance that should process the message. The consumer may be running in a cluster, and therefore have multiple instances; the eager assignment scheme identifies the instance of the consumer that should process the message. The natural implementation is then to queue the message onto the request queue, until the chosen consumer instance is available to do the work. When the consumer is available, the consumer requests a message from the queue and the message is dispatched. The eager assignment implementation can result in double-queuing, which occurs when a message is classified, a consumer instance is selected and the message is en-queued. The selection of consumer was made without consideration for the time at which that consumer will become ready to process another message, the consumer is likely to be processing its previously assigned message and the central processor cannot know for sure how long that will take. At best, the central processor has statistical knowledge of how long a message of the particular classification typically takes, on average. The disadvantage of such eager assignment of the consumer is that such a system prevents another instance of the consumer from processing the message, even if the other instance completes its previous work and becomes eligible to consume the message. The message is forced to wait for the assigned consumer instance to become eligible. The message is effectively double queued, once on the request queue and once again waiting for the assigned consumer instance. This is inefficient, because the eager assignment does not react to changes in background server load, availability or readiness, and prevents the message from being processed by an alternative consumer instance. To overcome this, the processor needs to be given greater discretion.
The lazy assignment implementation analyses each message on arrival and decides which classification fits the message. However, this assignment implementation does not identify the consumer instance. The message is en-queued, and awaits a consumer. The central processor monitors the loads on the backend servers and decides that a particular backend server is ready to accept work. The central processor dispatches that server, and that consumer instance requests a message. The problem that this can run into is that there are multiple consumer instances and they are competing for the messages. So when a consumer is dispatched, there is no guarantee that the message will still be available, as the message could have been consumed by another instance. This leads to poor utilisation of the backend servers, without tighter coupling between the classification and dispatch mechanisms the central processor cannot exercise sufficiently fine-grained and timely control of the workload dispatched to the backend servers.